{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pGUYKbJNWNgj"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "1PzPJglSWgnW"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b5P4BEg1XYd5"
      },
      "source": [
        "# TensorFlow Addons 优化器：ConditionalGradient\n",
        "\n",
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td><a target=\"_blank\" href=\"https://tensorflow.google.cn/addons/tutorials/optimizers_conditionalgradient\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\">在 TensorFlow.org 上查看</a></td>\n",
        "  <td><a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/addons/tutorials/optimizers_conditionalgradient.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\">在 Google Colab 中运行</a></td>\n",
        "  <td><a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/addons/tutorials/optimizers_conditionalgradient.ipynb\"><img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\">在 GitHub 上查看源代码</a></td>\n",
        "  <td><a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/addons/tutorials/optimizers_conditionalgradient.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\">下载笔记本</a></td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Faj8luWnYNSG"
      },
      "source": [
        "# 概述\n",
        "\n",
        "此笔记本将演示如何使用 Addons 软件包中的条件梯度优化器。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MrDjqjY6YRYM"
      },
      "source": [
        "# ConditionalGradient\n",
        "\n",
        "> 由于潜在的正则化效果，约束神经网络的参数已被证明对训练有益。通常，参数通过软惩罚（从不保证约束满足）或通过投影运算（计算资源消耗大）进行约束。另一方面，条件梯度 (CG) 优化器可严格执行约束，而无需消耗资源的投影步骤。它通过最大程度减小约束集中目标的线性逼近来工作。在此笔记本中，我们通过 MNIST 数据集上的 CG 优化器演示弗罗宾尼斯范数约束的应用。CG 现在可以作为 Tensorflow API 提供。有关优化器的更多详细信息，请参阅 https://arxiv.org/pdf/1803.06453.pdf\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dooBaYGLYYnn"
      },
      "source": [
        "## 设置"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qYo0FkL4O7io"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "TensorFlow 2.x selected.\n"
          ]
        }
      ],
      "source": [
        "import tensorflow as tf\n",
        "import tensorflow_addons as tfa\n",
        "from matplotlib import pyplot as plt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kR0PnjrIirpJ"
      },
      "outputs": [],
      "source": [
        "# Hyperparameters\n",
        "batch_size=64\n",
        "epochs=10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-x0WBp-IYz7x"
      },
      "source": [
        "# 构建模型"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4KzMDUT0i1QE"
      },
      "outputs": [],
      "source": [
        "model_1 = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(64, input_shape=(784,), activation='relu', name='dense_1'),\n",
        "    tf.keras.layers.Dense(64, activation='relu', name='dense_2'),\n",
        "    tf.keras.layers.Dense(10, activation='softmax', name='predictions'),\n",
        "])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XGADNG3-Y7aa"
      },
      "source": [
        "# 准备数据"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d6a-kbM_i1b2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11493376/11490434 [==============================] - 0s 0us/step\n"
          ]
        }
      ],
      "source": [
        "# Load MNIST dataset as NumPy arrays\n",
        "dataset = {}\n",
        "num_validation = 10000\n",
        "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
        "\n",
        "# Preprocess the data\n",
        "x_train = x_train.reshape(-1, 784).astype('float32') / 255\n",
        "x_test = x_test.reshape(-1, 784).astype('float32') / 255"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sOlB-WqjZp1Y"
      },
      "source": [
        "# 定义自定义回调函数"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8LCmRXUgZqyV"
      },
      "outputs": [],
      "source": [
        "def frobenius_norm(m):\n",
        "    \"\"\"This function is to calculate the frobenius norm of the matrix of all\n",
        "    layer's weight.\n",
        "  \n",
        "    Args:\n",
        "        m: is a list of weights param for each layers.\n",
        "    \"\"\"\n",
        "    total_reduce_sum = 0\n",
        "    for i in range(len(m)):\n",
        "        total_reduce_sum = total_reduce_sum + tf.math.reduce_sum(m[i]**2)\n",
        "    norm = total_reduce_sum**0.5\n",
        "    return norm"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "udSvzKm4Z5Zr"
      },
      "outputs": [],
      "source": [
        "CG_frobenius_norm_of_weight = []\n",
        "CG_get_weight_norm = tf.keras.callbacks.LambdaCallback(\n",
        "    on_epoch_end=lambda batch, logs: CG_frobenius_norm_of_weight.append(\n",
        "        frobenius_norm(model_1.trainable_weights).numpy()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qfhE1DfwZC1i"
      },
      "source": [
        "# 训练和评估：使用 CG 作为优化器\n",
        "\n",
        "只需用新的 TFA 优化器替换典型的 Keras 优化器 "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6-AMaOYEi1kK"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train on 60000 samples, validate on 10000 samples\n",
            "Epoch 1/10\n",
            "60000/60000 [==============================] - 5s 85us/sample - loss: 0.3745 - accuracy: 0.8894 - val_loss: 0.2323 - val_accuracy: 0.9275\n",
            "Epoch 2/10\n",
            "60000/60000 [==============================] - 3s 50us/sample - loss: 0.1908 - accuracy: 0.9430 - val_loss: 0.1538 - val_accuracy: 0.9547\n",
            "Epoch 3/10\n",
            "60000/60000 [==============================] - 3s 49us/sample - loss: 0.1497 - accuracy: 0.9548 - val_loss: 0.1473 - val_accuracy: 0.9560\n",
            "Epoch 4/10\n",
            "60000/60000 [==============================] - 3s 49us/sample - loss: 0.1306 - accuracy: 0.9612 - val_loss: 0.1215 - val_accuracy: 0.9609\n",
            "Epoch 5/10\n",
            "60000/60000 [==============================] - 3s 49us/sample - loss: 0.1211 - accuracy: 0.9636 - val_loss: 0.1114 - val_accuracy: 0.9660\n",
            "Epoch 6/10\n",
            "60000/60000 [==============================] - 3s 48us/sample - loss: 0.1125 - accuracy: 0.9663 - val_loss: 0.1260 - val_accuracy: 0.9640\n",
            "Epoch 7/10\n",
            "60000/60000 [==============================] - 3s 50us/sample - loss: 0.1108 - accuracy: 0.9665 - val_loss: 0.1009 - val_accuracy: 0.9697\n",
            "Epoch 8/10\n",
            "60000/60000 [==============================] - 3s 51us/sample - loss: 0.1081 - accuracy: 0.9676 - val_loss: 0.1129 - val_accuracy: 0.9647\n",
            "Epoch 9/10\n",
            "60000/60000 [==============================] - 3s 50us/sample - loss: 0.1065 - accuracy: 0.9675 - val_loss: 0.1058 - val_accuracy: 0.9683\n",
            "Epoch 10/10\n",
            "60000/60000 [==============================] - 3s 51us/sample - loss: 0.1039 - accuracy: 0.9683 - val_loss: 0.1126 - val_accuracy: 0.9646\n"
          ]
        }
      ],
      "source": [
        "# Compile the model\n",
        "model_1.compile(\n",
        "    optimizer=tfa.optimizers.ConditionalGradient(\n",
        "        learning_rate=0.99949, lambda_=203),  # Utilize TFA optimizer\n",
        "    loss=tf.keras.losses.SparseCategoricalCrossentropy(),\n",
        "    metrics=['accuracy'])\n",
        "\n",
        "history_cg = model_1.fit(\n",
        "    x_train,\n",
        "    y_train,\n",
        "    batch_size=batch_size,\n",
        "    validation_data=(x_test, y_test),\n",
        "    epochs=epochs,\n",
        "    callbacks=[CG_get_weight_norm])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8OJp4So9bYYR"
      },
      "source": [
        "# 训练和评估：使用 SGD 作为优化器"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SuizUueqn449"
      },
      "outputs": [],
      "source": [
        "model_2 = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(64, input_shape=(784,), activation='relu', name='dense_1'),\n",
        "    tf.keras.layers.Dense(64, activation='relu', name='dense_2'),\n",
        "    tf.keras.layers.Dense(10, activation='softmax', name='predictions'),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V8QC3xCwbfNl"
      },
      "outputs": [],
      "source": [
        "SGD_frobenius_norm_of_weight = []\n",
        "SGD_get_weight_norm = tf.keras.callbacks.LambdaCallback(\n",
        "    on_epoch_end=lambda batch, logs: SGD_frobenius_norm_of_weight.append(\n",
        "        frobenius_norm(model_2.trainable_weights).numpy()))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9BNi4yXGcDlg"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train on 60000 samples, validate on 10000 samples\n",
            "Epoch 1/10\n",
            "60000/60000 [==============================] - 3s 46us/sample - loss: 0.9498 - accuracy: 0.7523 - val_loss: 0.4306 - val_accuracy: 0.8844\n",
            "Epoch 2/10\n",
            "60000/60000 [==============================] - 2s 41us/sample - loss: 0.3851 - accuracy: 0.8916 - val_loss: 0.3298 - val_accuracy: 0.9068\n",
            "Epoch 3/10\n",
            "60000/60000 [==============================] - 3s 42us/sample - loss: 0.3230 - accuracy: 0.9064 - val_loss: 0.2917 - val_accuracy: 0.9150\n",
            "Epoch 4/10\n",
            "60000/60000 [==============================] - 2s 41us/sample - loss: 0.2897 - accuracy: 0.9169 - val_loss: 0.2676 - val_accuracy: 0.9241\n",
            "Epoch 5/10\n",
            "60000/60000 [==============================] - 3s 43us/sample - loss: 0.2658 - accuracy: 0.9237 - val_loss: 0.2485 - val_accuracy: 0.9288\n",
            "Epoch 6/10\n",
            "60000/60000 [==============================] - 2s 41us/sample - loss: 0.2467 - accuracy: 0.9301 - val_loss: 0.2374 - val_accuracy: 0.9285\n",
            "Epoch 7/10\n",
            "60000/60000 [==============================] - 3s 42us/sample - loss: 0.2308 - accuracy: 0.9343 - val_loss: 0.2201 - val_accuracy: 0.9358\n",
            "Epoch 8/10\n",
            "60000/60000 [==============================] - 2s 41us/sample - loss: 0.2169 - accuracy: 0.9388 - val_loss: 0.2096 - val_accuracy: 0.9388\n",
            "Epoch 9/10\n",
            "60000/60000 [==============================] - 2s 42us/sample - loss: 0.2046 - accuracy: 0.9421 - val_loss: 0.2009 - val_accuracy: 0.9404\n",
            "Epoch 10/10\n",
            "60000/60000 [==============================] - 2s 41us/sample - loss: 0.1939 - accuracy: 0.9448 - val_loss: 0.1900 - val_accuracy: 0.9442\n"
          ]
        }
      ],
      "source": [
        "# Compile the model\n",
        "model_2.compile(\n",
        "    optimizer=tf.keras.optimizers.SGD(0.01),  # Utilize SGD optimizer\n",
        "    loss=tf.keras.losses.SparseCategoricalCrossentropy(),\n",
        "    metrics=['accuracy'])\n",
        "\n",
        "history_sgd = model_2.fit(\n",
        "    x_train,\n",
        "    y_train,\n",
        "    batch_size=batch_size,\n",
        "    validation_data=(x_test, y_test),\n",
        "    epochs=epochs,\n",
        "    callbacks=[SGD_get_weight_norm])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1Myw0FVcd_Z9"
      },
      "source": [
        "# 权重的弗罗宾尼斯范数：CG 与 SGD"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0tJYQBRt-ZUl"
      },
      "source": [
        "CG 优化器的当前实现基于弗罗宾尼斯范数，并考虑将弗罗宾尼斯范数作为目标函数中的正则化器。因此，我们将 CG 的正则化效果与尚未采用弗罗宾尼斯范数正则化器的 SGD 优化器进行比较。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ewf17MW1cJVI"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f2cd4f5f6d8>"
            ]
          },
          "execution_count": 13,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXyTVRbw8d9pgbYIFmQpyFZAFqUb\nUAqylmUAFVFEUQaQdRwRRUYHHdx15B1HcEVfFRRRxhcVEEVFxQVkEWUtUgVlsZQdCpattEB73z9u\n0qSlS0qbpm3O9/PJJ8mT5MlpoKc357nPuWKMQSmllP8I8HUASimlSpcmfqWU8jOa+JVSys9o4ldK\nKT+jiV8ppfxMJV8H4InatWub8PBwX4ehlFLlyoYNG1KMMXVyby8XiT88PJz169f7OgyllCpXRGR3\nXtu11KOUUn5GE79SSvkZTfxKKeVnykWNXymAc+fOsXfvXtLT030dilJlSnBwMA0bNqRy5coePV8T\nvyo39u7dS/Xq1QkPD0dEfB2OUmWCMYajR4+yd+9emjZt6tFrtNSjyo309HRq1aqlSV8pNyJCrVq1\nivRNWBO/Klc06St1oaL+XmipRymlyghj4OxZe8nIsJfatSEoqGTfRxO/UkqVEmMgM9OV1DMycib5\ns2ftc9xVq1byiV9LPUoV0cGDB7ntttto3rw57du359prr+X3339n+/btDBgwIHt7z549WbFiRb77\nycjIoE+fPsTExPDBBx94/P7x8fEldib7tddeS2pqaonsq6waOnQoUVFRvPDCCyW638WLF/PMM89c\nsD0rC86cgdRUWLRoOb17D2DHDvjlF9i0CRISYOtWePrpF9m5M40//7R/DKpWhbAwaNIEWraEyEho\n1w5CQ0s0bMCLI34RmQ0MAA4bYyLctt8DTAAygc+NMQ94KwalSpoxhkGDBjFy5Ejef/99ADZv3syh\nQ4cYO3Ys06dPZ+DAgQAkJiayfv16unfvnue+Nm3aBEBCQsIFj2VmZhIYGOiln8JlyZIlXn8PTxhj\nMMYQEFCyY9GDBw+ybt06duzYUWL7dJZjevYcSOfOA9m3L+fI/dw513MPH7bb09PtqL16dXsdFAQL\nF77Igw8OJyysaonF5ilvjvjnAP3dN4hIT+AGINoY0waY7sX3VxXZpEkQH1+yl0mTCn3bZcuWUbly\nZe68887sbdHR0fz+++9cffXV2UkfICIiglGjRuW5n8OHDzN8+HDWrVtHTEwMO3fuJDw8nAcffJB2\n7doxf/58EhIS6NSpE1FRUQwaNIg///wz+/Vz584lJiaGiIgI1q5dC8Dp06cZM2YMcXFxtG3blk8+\n+QSAOXPmcNNNN9G/f39atGjBAw+4xlrh4eGkpKSQlJRERET2+Izp06fzxBNPAPDyyy9z1VVXERUV\nxW233ZbvZ/PEE08wZswY4uPjadasGS+//HL2Y88//zwRERFERETw4osvApCUlESrVq24/fbbiYiI\nYM+ePVSrVo3JkyfTpk0b+vTpw9q1a7P3t3jx4nzfOz09ndGjRxMZGUnbtm1ZtmwZAH379mXfvn3E\nxMSwcuXKPP8d2rdvD9g/4CJCcnIy589Ds2bN2bs3jcTEI1xzzWAiIzsQEdGB2bNXs2ULTJ8+h7vu\nupsDB2Dr1p3cdlsnbr01krlzHyE+vhqtW0Pz5hAYeIonnriZ669vzZQpw6hb1/Duuy9z4MB++vTp\nSc+ePcnMzGTUqFFEREQQGRlZ4t9OcvPaiN8Ys0JEwnNtHg88Y4zJcDznsLfeXylvSExMzE4U7n75\n5RfatWvn8X7q1q3Lm2++yfTp0/nss8+yt9eqVYuNGzcCEBUVxYwZM+jRowePPfYYTz75ZHbSTEtL\nIyEhgRUrVjBmzBgSExOZOnUqvXr1Yvbs2aSmphIXF0efPn0A+61i06ZNBAUF0apVK+655x4aNWrk\nUazPPPMMf/zxB0FBQYWWhbZt28ayZcs4efIkrVq1Yvz48fz888+8/fbb/PTTTxhj6NixIz169KBm\nzZps376dd955h06dOgH2j1evXr2YNm0agwYN4pFHHuHrr7/m119/ZeTIkTn+sLp79dVXERG2bNnC\ntm3b6Nu3L7///juLFy9mwIABF3yrMsaOzENC6nL6dDrbtp3go49W0qZNLHPnriQqqivVqtXl4MGq\nPPLIOG699R906tSVlJRkxozpxw8/bKVePTh0yJZjBg68lwcfvJehQ4fy+uuvA7Y2X6mS/Wb3yy+/\ncPnll9OlSxdWr17NxIkTef7551m2bBm1a9dmw4YN7Nu3j8TERACvl99K++BuS6CbiEwF0oF/GmPW\n5fVEEbkDuAOgcePGpRehKh8cCbCsGjRoENu3b6dly5Z89NFHHr/u1ltvBeD48eOkpqbSo0cPAEaO\nHMktt9yS/byhQ4cC0L17d06cOEFqaipLly5l8eLFTJ9uv0inp6eTnJwMQO/evQl1FIuvuuoqdu/e\n7XHij4qKYtiwYdx4443ceOONBT73uuuuIygoiKCgIOrWrcuhQ4dYtWoVgwYN4pJLLgHgpptuYuXK\nlQwcOJAmTZpkJ32AKlWq0L+/LRRERkYSFBRE5cqViYyMJCkpKd/3XbVqFffccw8ArVu3pkmTJmzb\n9jshIZeSlWVLLs4DqOnp9tp5ELV1684sXbqaH39cwfjxD7F69ZeEhhri47tx1VWwceM3HDz4Ky+9\nZJ+flnaCkJBThITYxB4QAGvWrOHjjz8G4K9//Sv//Oc/s2OLi4ujYcOGAMTExJCUlETXrl1zxN+s\nWTN27drFPffcw3XXXUffvn0L+VcpntI+uFsJuAzoBEwGPpR8JqAaY2YaY2KNMbF16lzQTlopn2jT\npg0bNmzIc7tzpA6waNEi5syZw7Fjx4q0f2dyLEzuXxsRwRjDwoULSUhIICEhgeTkZK688koAgtym\nhQQGBnL+/Pkcr69UqRJZWVnZ991PBvr888+ZMGECGzdupEOHDhe81l1h75Nb7p+3cuXK2T9bQEBA\n9v4CAgLy3FdmJqSl2dH7sWOwezf8/jucPm0PoP72m03yyclw5Ii9HRwMdetC48b2IOrAgd05cGAl\nqam7GT/+Bnbt2kxi4ir+8pduVK0KWVlZ/Pjjj9mf6759+6hWrVqBP1dRP5OaNWuyefNm4uPjef31\n1xk3bpzH+78YpZ349wIfGWstkAXULuUYlLpovXr1IiMjg5kzZ2Zv+/nnn2nZsiWrV6/OUYdOS0u7\n6PcJDQ2lZs2a2XXpuXPnZo/+gexZQKtWrSI0NJTQ0FD69evHjBkzMI6hrPPgsSfCwsI4fPgwR48e\nJSMjI7v8lJWVxZ49e+jZsyf//e9/OX78OKdOnSrSz9KtWzc+/vhj0tLSOH36NIsWLaJbt24evdZZ\nknG+5f798McfsG0bbN5sZ8n8+iu0aNGNuXPf488/YefO3zl0KJmOHVvRsKE9kBoVBW3bQps2cMUV\n0KiRTf6XXgq9enXjvff+R4sWLQgICOCyyy5jyZIl2aPyvn37MmPGjOyY8joY36lTJxYuXAiQfdC/\nMNWrV+fkyZMApKSkkJWVxeDBg3n66adzDCK8obRLPR8DPYFlItISqAKklHIMSl00EWHRokVMmjSJ\n//73vwQHBxMeHs6LL77IZ599xn333cekSZMICwujevXqPPLIIxf9Xu+88w533nknaWlpNGvWjLff\nfjv7seDgYNq2bcu5c+eYPXs2AI8++iiTJk0iKiqKrKwsmjZtmuP4QUEqV67MY489RlxcHA0aNKB1\n69aAnV00fPhwjh8/jjGGiRMnUqNGjSL9HO3atWPUqFHExcUBMG7cONq2bZtdunEmd2f5Ze9ee/vI\nETh+3CZ4sNMk9++HKlVsMg8Ndc2QeeSRu7jvvvGMGBFJpUqV+N//5tCsWRBJSbYUU6VK/vGFh4dj\njMmefdW1a1f27t1LzZo1AXtwe8KECURFRXH+/Hm6d++eXcd3evHFFxk+fDhTp06lf//+2WW1gtxx\nxx3079+fyy+/nBdffJHRo0dnf+v6z3/+U4RPuOjE5D5boKR2LDIPiMeO6A8BjwNzgdlADHAWW+P/\nrrB9xcbGGl2BS23dujW7dKHKF/fk7qyxO68zMmxSdxJxJffgYFdyd15KeMZniUhLSyMkJAQR4f33\n32fevHnZs6pKS16/HyKywRgTm/u53pzVMzSfh4Z76z2VUr7jTO7uCb2g5O5M5JdemjPJV6liHy9P\nNmzYwN13340xhho1amR/CyurtGWDUl729ttv85JzSohDly5dePXVV30U0cUzBmbNeptXXnmJrCx7\nPysLoqO7MHnyqznaDTiTe3BwySX3r776igcffDDHtqZNm7Jo0aJCXzthwgRWr16dY9u9997L6NGj\nix5ILt26dWOzsyZVDnit1FOStNSjQEs9pcV5Zmp+ZZncyd29HON+uzyO3MuzMlHqUUqVXbnLMunp\n+Sf3gABXUq9RI2eSr1xZk3t5pIlfqQrs/Pmcid090bvX3PNK7s7Ruyb3ikcTv1LlXFZWzoOp7pfc\n5wo5E3q1avbaedHk7l808StVDriXZnKP3DMycj63cmWb4GvUcCX2sjwVUpU+/W+gVBFNnTqVNm3a\nEBUVRUxMDD/99BPnz5/noYceokWLFsTExBATE8PUqVOzXxMYGEhMTAxt2rQhOjqa5557LkeLBKfz\n5+1ZqikpcOONQ2ndOooHH3yBTZvg559tO4LkZPv4uXO2h3v9+tC0KVx5Jbz88ii2b19A69YQHg71\n6tk/ACEhRU/648aN49dffy3mp1W2OTuBTp48uUT3u379eiZOnFjgc3J3RHU3Z84c9u/fX6IxudMR\nv1JFsGbNGj777DM2btxIUFAQKSkpnD17lkceeYSDBw+yZcsWgoODOXnyJM8991z260JCQrJP9T90\n6DBDh/6Vw4dPMGnSkzlKNM7STErKQTZsWMeSJTtylGSCgqBSpfOEhFTKszRTkiP6N998s+R2Vkze\nWp9g5syZHDt2rMT3HRsbS2zsBZNpPDZnzhwiIiK4/PLLSzAqFx3xq3LJR+34OXDgALVr185uvFW7\ndm1q1KjBrFmzmDFjBsHBwYDtw/Loo09w+jQcPWpLNTt32lWY9u6ty8SJM3njjVfYu9dw/Litr9es\nCQ0b2l4ykyf3JSVlHyNGxHDw4EpGjIhn6tRJ9OoVy2uvvcTu3Un06tWLqKgoevfund2FE+Cbb74h\nNjaWli1bZrdsyMzMZPLkyXTo0IGoqCjeeOMNAJYvX058fDw333wzrVu3ZtiwYdm9ftxX+nJvSrZg\nwYLsdQbmz59PREQE0dHR+S44AwWvCTBv3jwiIyOJiIjIMUe/WrVq3H///URHR7NmzRrCw8OZMmUK\nMTExxMbGsnHjRvr160fz5s0vaKHgzhjD5MmTs3vdO/scDRw4kFOnTtG+ffs8V0DLzMykadOmGGNI\nTU0lMDAwe0W17t27s3379nzXQFi+fDkDBgwA4MiRI/zlL3+hTZs2jBs3jiZNmpCSkpL9Hn/7299o\n06YNffv25cyZMyxYsID169czbNgwYmJiOHPmDP/617+y10Rw7/x5sXTEr1QR9O3bl6eeeoqWLVvS\np08fbr31VqpVq0nDho1JT69Oaqpddi893c6FdzLGdpEMCbE9Zpo0aQZkcvnlh2nQIOyC9/n00wv7\nyJ89ezY7EV9//fWMHDmSkSNHMnv2bCZOnJjdFjgpKYm1a9eyc+dOevbsyY4dO3j33XcJDQ1l3bp1\nZGRk0KVLl+zWv3n1i8/dNjg/Tz31FF999RUNGjQotId8XmsCBAYG8uCDD7JhwwZq1qxJ3759+fjj\nj7nxxhs5ffo0HTt2zPHNqXHjxiQkJPCPf/yDUaNGsXr1atLT04mIiMixOI67jz76iISEBDZv3kxK\nSgodOnSge/fuLF68mGrVquXZdA1sea5Vq1b8+uuv/PHHH7Rr146VK1fSsWNH9uzZQ4sWLXjooYfy\nXQPB6cknn6RXr15MmTKFL7/8krfeeiv7se3btzNv3jxmzZrFkCFDWLhwIcOHD+eVV15h+vTpxMbG\ncvToURYtWsS2bdsQkRLp1a+JX5VLpd2O33lSU2ZmNT7/fAMrV65k5cplDB58K6NGPUR6um0JHBAA\nX3zxNu+99xKpqUf56qsfuOKKRgQE2DVUc6tUhN9AZ69+sCUnZ5//ESNG5BhBDxkyhICAAFq0aEGz\nZs3Ytm0bS5cu5eeff2bBggWA7fe/fft2qlSp4lG/+Px06dKFUaNGMWTIEG666aYCn5vXmgBHjx4l\nPj4eZ+v1YcOGsWLFCm688UYCAwMZPHhwjn04F2KJjIzk1KlTVK9enerVq2cvEpNXA7lVq1YxdOhQ\nAgMDCQsLo0ePHqxbty7fRV3cdevWjRUrVvDHH38wZcoUZs2aRY8ePejQoQNAgWsguL+/88zi/v37\nZzd/A3vWcUxMDADt27fPc82B0NBQgoODGTt2LAMGDMj+JlEcmviVcpOVdeGUSOcI3nVSUyCtW8cT\nExNPREQk77//BkeOJBMefpJatarTrt1oHn54NBEREVSvnklIyIXvs2vXLgIDA6lbt67HsRW3V/+M\nGTPo169fjseWL1/uUb9493269+p//fXX+emnn/j8889p3749GzZsoFatWnnGVdRe/cHBwRfU3t37\n87vvL79+/cXVvXt3XnvtNfbv389TTz3FtGnTWL58eXZbaecaCK1atcrxukOHDnm0/9yfyZkzZy54\nTqVKlVi7di3ffvstCxYs4JVXXuG77wrtbVkgrfErv+Q+e2bPHti+HbZsgY0bbX/3XbtsC+DTp23r\ngbp1oUkTCAj4jUsu2U5MDLRuDfv2JRAZ2Ypx48Zy//13k5Fhk2JmZiZn3Ws9bo4cOcKdd97J3Xff\nfUGS9lTnzp2z+76/9957Ofrbz58/n6ysLHbu3MmuXbto1aoV/fr147XXXuOcYyXw33//ndOnT3v8\nfmFhYWzdupWsrKwcfXF27txJx44deeqpp6hTpw579uwp0s8RFxfH999/T0pKCpmZmcybNy/HugMl\noVu3bnzwwQdkZmZy5MgRVqxYkd0i2pP4fvjhBwICAggODiYmJoY33ngj+3iGJ2sgdOnShQ8//BCw\n3xDc107Oj3uv/lOnTnH8+HGuvfZaXnjhhRLpCaQjflWhOUfwZ87kvLjnZGe/mapV4bLLbB3eOYMm\n92SP5ORTjB17D6mpqVSqVIkrrriCmTNnEhoayqOPPuoY5VcnJCSEkSNHZs/KOHPmDDExMZw7d45K\nlSoxYsQI7rvvvov+uWbMmMHo0aOZNm0aderUydGrv3HjxsTFxXHixAlef/11goODGTduHElJSbRr\n1w5jDHXq1Mk+JuCJZ555hgEDBlCnTh1iY2OzF2OZPHky27dvxxhD7969iY6OLtLPUb9+fZ555hl6\n9uyJMYbrrruOG264oUj7KMygQYNYs2YN0dHRiAjPPvss9erV8+i1QUFBNGrUKHt5yG7dumUfjAbP\n1kB4/PHHGTp0KHPnzuXqq6+mXr16VK9evcAFbUaNGsWdd95JSEgIX3zxBTfccAPp6ekYY3j++ecv\n8pNw0SZtqtwoqEmbMfZEprS0nInerSqRneBDQlwXZ4LXs1aVt2RkZBAYGEilSpVYs2YN48ePz/eA\ncnFokzZVYTkPsqan2yTvTO5nzuRsLBYUZBN7zZquJK9nripfSE5OZsiQIWRlZVGlShVmzZrl65A0\n8auy68gRSEy0tffERBg61CZ49xNeq1RxLZ7tPor3wrk+XlGc/vJlkS9/ni1btjBixIgc24KCgvjp\np58Kfe3UqVOZP39+jm233HILDz/8cLHjatGiRZHWPy4NWupRPnfypD2xyT3JJybC4cOu51x2Gcyf\nv5XmzVsTEiLZSb4o0yGVqqiMMWzbtk1LParsOX8efvsNEhJyJvndu13PqVoVIiJgwAB77bzUqwdJ\nScFccslRatWqddGzYZSqaIwxHD16NPuscU9o4ldeceqUTeybNtlEn5Bg7zsPtlaqZKdDdu4Md9zh\nSvDh4fnX4Rs2bMjevXs5cuRIqf0cSpUHwcHB2SfheUITvyq2gwddyT0hwSb77dtdB1tr1oS2beGu\nu+x1dDS0amXr80VRuXJlmjZtWvI/gFJ+RhO/8lhWFuzY4UruzkR/8KDrOeHhNrkPGwYxMfZ2w4Y6\nXVKpskQTv8pTerqtv7sn+M2b7ZmsYEs1bdpAv36uBB8dbXu/K6XKNk38iqNHLyzVbNsGmZn28Usv\ntcl97Fh7HRMDV11l58UrpcqfQhO/iNwCfGmMOSkijwDtgKeNMRu9Hp0qUcZAUlLOBJ+QYHvVODVs\naBP7oEF2FB8TU/ABV6VU+ePJiP9RY8x8EekK9AGmAa8BHb0amSq2jAzYsAFWrbKX1avh2DH7WECA\nnVXTvbtrFB8TA7Vr+zZmpZT3eZL4HV/4uQ6YaYz5XESe9mJM6iKlpsKaNbBypU30a9e6FuJu3dqO\n4uPi7Eg+IoI82wUrpSo+TxL/PhF5A/gL8F8RCULbOZcJe/a4RvOrVtl58sbYA6/t28Pdd0PXrtCl\nCzjWuVBKKY8S/xCgPzDdGJMqIvWBkl2SXhUqK8v2iXdP9M4zXqtVsydC3XyzTfRxceDhmh1KKT/k\nSeJ/wxiT3fnIGHNARJ4FlnovLJWRAevX56zPO9dvqFcPunWD++6z15GR2rNGKeU5T9JFG/c7IhII\ntPdOOP4rNRV++MFVn1+3Lmd9fvBgm+S7doWmTfWEKKXUxcs38YvIFOAhIERETjg3A2eBmaUQW4WW\nnJyzbJOYmLM+f889Nsl37qz1eaVUyco38Rtj/gP8R0T+Y4yZUooxVThZWbbtsHuiT062j1WvbpP7\nkCGu+nzVqr6NVylVsRVa6jHGTBGRBkAT9+cbY1Z4M7CKYOdOmD0b3nkH9u2z2+rXtyWbf/7TJnqt\nzyulSpsnZ+4+A9wG/IprTr8BNPHnIS0NFi6Et96C77+3J0r17w9PP21PltL6vFLK1zwZaw4CWhlj\nMrwdTHlljD0YO3s2zJsHJ05A8+YwdSrcfrttg6CUUmWFJ4l/F1AZ0MSfy5Ej8L//2YSfmGjPhL35\nZtvMrFs37W+jlCqbCprVMwNb0kkDEkTkW9ySvzFmovfDK3syM+Grr2yyX7wYzp2zB2Rffx1uuw1C\nQ30doVJKFaygEb9zdfMNwOKi7lhEZgMDgMPGmAjHtieAvwHOtfMeMsYsKeq+fWHnTnj7bZgzxx6o\nrV3btkQYM8b2vVFKqfKioOmc7xRz33OAV4B3c21/wRgzvZj7LhXOA7WzZ8Py5bZ0068fvPQSXH99\n0ZcOVEqpssCTWT1bsCUfd8ex3wieNsYczet1xpgVIhJe3ABLW14Haps1s7NyRo7UA7VKqfLPk4O7\nX2Cncf4/x/3bgKrAQeyo/voivufdInI79g/H/caYP/N6kojcAdwB0Lhx4yK+RdGlpNgDtW+9lfNA\n7ZgxdhqmHqhVSlUUYkzuwXyuJ4hsNMa0y2ubiGwxxkQW8Npw4DO3Gn8YkIL9BvFvoL4xZkxhQcbG\nxpr169cX9rQiy8yEpUvt6P6TT+yB2g4d7KwcPVCrlCrvRGSDMSY293ZPRvyBIhJnjFnr2FEHINDx\n2PmiBGGMOeQW0Czgs6K8vqTkPlBbqxZMmGBH95H5/hlTSqmKwZPEPw6YLSLVsE3aTgDjROQS4D9F\neTMRqW+MOeC4OwhILMrri0MP1CqllOVJr551QKSIhDruH3d7+MP8Xici84B4oLaI7AUeB+JFJAZb\n6kkC/n7RkXvAGNvT/q239ECtUko5FXQC13BjzP9E5L5c2wEwxjxf0I6NMUPz2PzWxQR5scaPhzfe\ngOBg1xm1eqBWKeXvChrxOxfvq14agXjDzTdDTAwMHaoHapVSyqnQWT1lgbdm9SilVEWW36yeQose\nItJSRL4VkUTH/SgRecQbQSqllPI+T6rds4ApwDkAY8zP2JO4lFJKlUOeJP6qzjn8boo0f18ppVTZ\n4UniTxGR5jj69YjIzcCBgl+ilFKqrPLkBK4JwEygtYjsA/4Ahnk1KqWUUl7jyQlcu4A+jjN1A4wx\nJ70fllJKKW/xZFbPThF5DxgBeL9NplJKKa/ypMZ/FfAGUAuY5vhDsMi7YSmllPIWTxJ/JnYqZyaQ\nBRx2XJRSSpVDnhzcPQFsAZ4HZuW34pZSSqnywZMR/1BgBXAX8L6IPCkivb0bllJKKW/xZFbPJ8An\nItIauAaYBDwAhHg5NqWUUl7gyayehSKyA3gJu9bu7UBNbwemlFLKOzyp8f8H2GSMyfR2MEoppbzP\nk1KP9kNWSqkKRNeiUkopP5Nv4heRLo7roNILRymllLcVNOJ/2XG9pjQCUUopVToKqvGfE5GZQAMR\neTn3g8aYid4LSymllLcUlPgHAH2AfsCG0glHKaWUt+Wb+I0xKdgzdbcaYzaXYkxKKaW8yJNZPUdF\nZJGIHHZcFopIQ69HppRSyis8SfxvA4uByx2XTx3blFJKlUOeJP66xpi3jTHnHZc5QB0vx6WUUspL\nPF1sfbiIBDouwwFtzayUUuWUJ4l/DDAEOAgcAG4GRnszKKWUUt7jSa+e3cDAUohFKaVUKdBePUop\n5Wc08SullJ/RxK+UUn6m0Bq/iNTArroV7v587dWjlFLlkycrcC0BfgS2AFneDUcppZS3eZL4g40x\n93k9EqWUUqXCkxr/XBH5m4jUF5HLnBevR6aUUsorPBnxnwWmAQ8DxrHNAM28FZRSSinv8STx3w9c\n4WjTrJRSqpzzpNSzA0jzdiBKKaVKhycj/tNAgogsAzKcGwubzikis7GreB02xkTkeux+YDpQR79J\nKKVU6fIk8X/suBTVHOAV4F33jSLSCOgLJF/EPpVSShVTgYlfRAKBvsaYYUXdsTFmhYiE5/HQC8AD\nwCdF3adSSqniK7DGb4zJBJqISJWSeDMRuQHY58kaviJyh4isF5H1R44cKYm3V0ophWelnl3AahFZ\njK33A2CMeb4obyQiVYGHsGWeQhljZgIzAWJjY00hT1dKKeUhTxL/TsclAKhejPdqDjQFNosIQENg\no4jEGWMOFmO/SimlisCThX4vo4AAABPLSURBVFieBBCRao77py7mjYwxW4C6zvsikgTE6qwepZQq\nXYXO4xeRCBHZBPwC/CIiG0SkjQevmwesAVqJyF4RGVv8cJVSShWXJ6WemcB9xphlACISD8wCOhf0\nImPM0EIeD/csRKWUUiXJkzN3L3EmfQBjzHLgEq9FpJRSyqs8mtUjIo8Ccx33h2Nn+iillCqHPBnx\njwHqAB85LnUc25RSSpVDnszq+RPQZRaVUqqC8GTN3ZbAP7lwzd1e3gtLKaWUt3hS458PvA68CWR6\nNxyllFLe5kniP2+Mec3rkSillCoVnhzc/VRE7tI1d5VSqmLwZMQ/0nE92W2brrmrlFLllCezepqW\nRiBKKaVKhyelHqWUUhWIJn6llPIzmviVUsrPeNKWuYuIXOK4PVxEnheRJt4PTSmllDd4MuJ/DUgT\nkWjgfuxqXO96NSqllFJe40niP2+MMcANwCvGmFcp3hKMSimlfMiTefwnRWQKth1zdxEJACp7Nyyl\nlFLe4smI/1YgAxjrWBS9ITDNq1EppZTyGk9O4DoIPO92Pxmt8SulVLnlSVvmk9gWDQBVsGWeU8aY\nUG8GppRSyjs8GfFnH8gVEcEe5O3kzaCUUkp5T5FO4DLWx0A/L8WjlFLKyzwp9dzkdjcAiAXSvRaR\nUkopr/JkOuf1brfPA0nYco9SSqlyyJMa/+jSCEQppVTpyDfxi8gDxphnRWQGrlk92YwxE70amVJK\nKa8oaMS/1XG9vjQCUUopVTryTfzGmE8d1++UXjhKKaW8zZNZPS2BfwLh7s83xvTyXlhKKaW8xZNZ\nPfOB14E3gUzvhqOUUsrbPEn8540xr3k9EqWUUqXCkzN3PxWRu0Skvohc5rx4PTKllFJe4cmIf6Tj\nerLbNgM0K/lwlFJKeZsnJ3A1LY1AlFJKlQ5PFluvKiKPiMhMx/0WIjLA+6EppZTyBk9q/G8DZ4HO\njvv7gKe9FpFSSimv8iTxNzfGPAucAzDGpAHi1aiUUkp5jSeJ/6yIhODo1yMizbFr8CqllCqHPEn8\njwNfAo1E5D3gW+CBwl4kIrNF5LCIJLpt+7eI/CwiCSKyVEQuv+jIlVJKXZRCE78x5mvgJmAUMA+I\nNcYs92Dfc4D+ubZNM8ZEGWNigM+Ax4oSrFJKqeLzZB4/QA+gK7bcUxlYVNgLjDErRCQ817YTbncv\nIY92z0op5deOHIFNmyAhwV4efxxatSrRt/CkSdv/Ba7AjvYB/i4ifYwxEy7mDUVkKnA7cBzoWcDz\n7gDuAGjcuPHFvJVSSpVdWVmwa5dN7u6Jfv9+13MaN4Z9+0o88YsxBQ+6RWQbcKVxPFFEAoBfjDFX\nFrpzO+L/zBgTkcdjU4BgY8zjhe0nNjbWrF+vywIopcqpjAz45ZecCX7zZjh50j4eGAhXXglt20JM\njL1ER0OtWsV6WxHZYIyJzb3dk1LPDqAxsNtxv5FjW3G9ByzBHjxWSqmK4dgxm9TdR/Jbt8L58/bx\natVsUr/9dpvg27aFNm0gOLjUQixo6cVPsTX46sBWEVnreCgOWJvf6woiIi2MMdsdd28Atl3MfpRS\nyueMgeTknKP4hATYvdv1nPr1bXIfMMA1mm/eHAI8mVDpPQWN+KcXZ8ciMg+IB2qLyF7syP5aEWkF\nZGG/QdxZnPdQSqlSce6cHbU7k7sz2aem2sdFoGVLuPpqGD/eVa4JC/Nt3PkoaOnF7523RSQM6OC4\nu9YYc7iwHRtjhuax+a0iR6iUUqXpxAn4+eecCT4xEc6etY8HB0NUFAwZ4irVREbCJZf4Nu4i8GRW\nzxBgGrAc26phhohMNsYs8HJsSinlPcbYGTPuZZqEBNi50/WcWrVsYr/3XtcovmVLqOTpTPiyyZPo\nHwY6OEf5IlIH+AbQxK+UKh/OnYNt21wHXZ2Xo0ddz7niCpvkR4+2B1/btoXLL7dlnArGk8QfkKu0\ncxTPWj0opVTpO37cVapxXtxLNUFBtjQzaJBrFB8VBdWr+zbuUuRJ4v9SRL7CdQLXrdhpmEop5TvG\nwN69F5Zqdu1yPad2bVepJjraJvlWrcp9qaa4PFmBa7KI3IRt2QAw0xhTaMsGpZQqMc5STe4kf+yY\n6zktWkD79jB2rGskX79+hSzVFFeBiV9EAoFvjDE9gY9KJySllF8rrFQTHGxLNYMHuxJ8ZKRflWqK\nq8DEb4zJFJEsEQk1xhwvraCUUn7k2DFYtgy+/Ra++w5++831WJ06FXJWja958umdAraIyNfAaedG\nY8xEr0WllKq40tJg1Sqb6L/5xs6VN8a2MujRw7YycJ7lWq+elmq8wJPE/xHltcxz/jwcPAgNG/o6\nEqX817lzsG6dTfTffgtr1tiyTeXK9kzXJ56A3r0hLs5uU15XUK+exsaYZGPMO6UZUIm66y744gtY\nuRLCw30djVL+wRhbk3cm+u+/t10oRewofuJEm+i7dStXZ7tWJAWN+D8G2gGIyEJjzODSCakETZgA\n8+dDr142+Tdo4OuIlKqYkpJcif7bb+Gw49SfK66Av/4V+vSBnj2L3WZYlYyCEr97Ya2ZtwPxiuho\n+Oor+5+ud2878iijTZOUKleOHHEdkP3mG9fc+bAw+/vm/J3TRZTKpIISv8nndvkSFwdLlkC/fvCX\nv9j/rDrqUKpoTp2y35q/+cYm+82b7fZLL4X4eDvrpndvuOoqPRhbDhSU+KNF5AR25B/iuI3jvjHG\nXOr16EpK167wySe2J3b//vY/b2ior6NSquw6exbWrnUl+h9/tJMlqlSBLl3g6adtoo+N1amV5VBB\nbZkDSzMQr+vTBxYutP05rr3WloCqVfN1VEr5XlaWXRrwt99cNfoVK+D0aTt6b98e7r/fJvouXaBq\nVV9HrIrJv/5UX3cdzJtn+2jfcAN89hmEhPg6KuXPzp+3qzilp9vkW9Dl7NnCn3MxF+eSgE6tWsHI\nkTbRx8fDZZf55KNR3uNfiR/sad7vvGNPEhk8GBYtst36lCothw7Bl1/aqcZLl8Kff17cfgIC7P/d\nwi7Vqnn2vAYN7MwbPe+lwvO/xA8wfLg9e/Dvf4ehQ+HDD7VOqbwnMxN++skm+iVLYONGu71ePfvN\ns2tXz5Oz+yWwYlVjVenx32x3xx1w5gxMmmS/1r77rv4iqZJz6JA9jrRkiWtUHxBgz1SdOhWuucZO\nN/bxotvKP/lv4gc7Be3MGZgyxdb6Z87UX0R1cdxH9V98ARs22O3OUf0119jpxDVr+jZOpfD3xA/w\nr3/Zss+//22T/8sv6zxk5RnnqP6LL+y1+6j+6adtso+J0cGEKnM08QM8+aRN/s89Z6eqPfOMJn91\nocxMO7fdWat3jurDwmDgQNeoXmfBqDJOEz/YJD9tmk3+zz5rG0c99pivo1JlweHDOWfgHDtmR/Cd\nOumoXpVbmvidROCVV2zN//HHbdln8mRfR6VKm/uo/osvYP16uz0sDK6/Xkf1qkLQxO8uIADefNMm\n/wcesGWfCRN8HZXytsOHc9bq3Uf1//63TfZt2+qoXlUYmvhzCwyEuXPtmZR3321H/mPG+DoqVRKy\nsuwB2aQk2L0bfvnFlnE2bLA95OvWtf2crr1WR/WqQtPEn5fKleGDD+wBu3HjbPIfOtTXUanCZGXZ\nFdeSklyX3btz3s7IcD3fOap/6ikd1Su/ook/P0FBtp3DtdfCiBEQHGwbvCnfycqCAwdyJnb3pL57\nt+1n465OHbv6WnS0nU8fHm4vTZrYa10BSvkhTfwFqVoVPv0U+vaFW2+1rZ2vucbXUVVcmZmwf3/O\nUbr7JTnZrt/qLizMJvG2be0fZmdiDw+3i4BoYlfqApr4C1O9uj3o16sX3HQTfP65va0uTmamXXh7\n+/a8E3vuTpH16tkkHhsLN998YWLXFsFKFZkmfk/UqGHncMfH27r/0qXQubOvoyo/srJg9WrbDG/B\nAluHd6pf3ybxjh1tu+zciV3bZitV4jTxe6p2bbsaUffuttzz3Xd2gQqVt6wsu2rThx/aBe/377fH\nSa67zo7c27WziT042NeRKuV3NPEXRb16dnWibt1s3X/5coiM9HVUZYcx9uQnZ7Lfs8ceJL/mGjua\nv/56XfVMqTJAE39RNWpkR/vdutnlHFessCsW+StjbH/5Dz6wCX/3bjsdtn9/+D//x5bGLi0/yzMr\n5Q808V+MZs3syL9HD7s83YoVdpu/MAY2b3Yl+1277EI2ffvahnc33GCPiyilyiRN/BerdWv4+mu7\nVJ0z+Tdq5OuovMcYSEx0Jfvt2+1Zzr17w8MPw4036pmuSpUTmviLIyrK9nbp3duV/OvV83VUJevX\nX22i/+AD2LbNntnas6dtYDdokD3orZQqVzTxF1dsrJ3n37evrfkvX17+k+Fvv9lk/+GHdpQvYsta\n995rz2WoW9fXESqlikETf0no3BkWL7ZTFfv2tQd/y1uNe8cOV7LfvNkm+65dbavqwYMr3jcZpfyY\n1xK/iMwGBgCHjTERjm3TgOuBs8BOYLQxJtVbMZSqXr3go49c66suXWrP+i3L/vjDlew3brTbOneG\nF1+0c+0bNPBtfEopr/BmK8I5QP9c274GIowxUcDvwBQvvn/pu+YaWwtft85OY0xL83VEF0pOhunT\nIS7OzkT617/s9MvnnrOPrV5tSzqa9JWqsLw24jfGrBCR8Fzblrrd/RG42Vvv7zODBsG778Lw4bYe\n/skn9iQmbzt3zi4ocuiQ6zr37f37bQ96sMcmnn0WbrnFtkdQSvkNX9b4xwAf5PegiNwB3AHQuHHj\n0oqpZPz1r3Yhl7FjbVfP+fPtqLqo0tLyTuB5Jfdjx/LeR3Cw7WAZFmZH+MOG2bNomzcv3s+olCq3\nfJL4ReRh4DzwXn7PMcbMBGYCxMbGmlIKreSMGWMT9z332H7+771np0IeP15wAnd/7PTpvPcdGmoT\ned260KaNPb5Qt64rwTsfCwuzLRJESvdnV0qVaaWe+EVkFPagb29jTPlL6EVx992u9Xu//RZOnLhw\noRCwibl2bVfC7tjxwgTuvF23rjY2U0oVS6kmfhHpDzwA9DDGlMEjn14webKd2rl6dd7JPCwMatWy\nLQ+UUqoUiLcG3SIyD4gHagOHgMexs3iCgKOOp/1ojLmzsH3Fxsaa9evXeyVOpZSqqERkgzEmNvd2\nb87qyWt18re89X5KKaU84815/EoppcogTfxKKeVnNPErpZSf0cSvlFJ+RhO/Ukr5GU38SinlZzTx\nK6WUn/HaCVwlSUSOALsv8uW1gZQSDKe808/DRT+LnPTzyKkifB5NjDF1cm8sF4m/OERkfV5nrvkr\n/Txc9LPIST+PnCry56GlHqWU8jOa+JVSys/4Q+Kf6esAyhj9PFz0s8hJP4+cKuznUeFr/EoppXLy\nhxG/UkopN5r4lVLKz1ToxC8i/UXkNxHZISL/8nU8viIijURkmYj8KiK/iMi9vo6pLBCRQBHZJCKf\n+ToWXxORGiKyQES2ichWEbna1zH5ioj8w/F7kigi80Skwq11WmETv4gEAq8C1wBXAUNF5CrfRuUz\n54H7jTFXAZ2ACX78Wbi7F9jq6yDKiJeAL40xrYFo/PRzEZEGwEQg1hgTAQQCt/k2qpJXYRM/EAfs\nMMbsMsacBd4HbvBxTD5hjDlgjNnouH0S+0vdwLdR+ZaINASuA970dSy+JiKhQHccK+QZY84aY1J9\nG5VPVQJCRKQSUBXY7+N4SlxFTvwNgD1u9/fi58kOQETCgbbAT76NxOdeBB4AsnwdSBnQFDgCvO0o\nfb0pIpf4OihfMMbsA6YDycAB4LgxZqlvoyp5FTnxq1xEpBqwEJhkjDnh63h8RUQGAIeNMRt8HUsZ\nUQloB7xmjGkLnAb88piYiNTEVgaaApcDl4jIcN9GVfIqcuLfBzRyu9/Qsc0viUhlbNJ/zxjzka/j\n8bEuwEARScKWAHuJyP98G5JP7QX2GmOc3wIXYP8Q+KM+wB/GmCPGmHPAR0BnH8dU4ipy4l8HtBCR\npiJSBXuAZrGPY/IJERFs/XarMeZ5X8fja8aYKcaYhsaYcOz/i++MMRVuVOcpY8xBYI+ItHJs6g38\n6sOQfCkZ6CQiVR2/N72pgAe6K/k6AG8xxpwXkbuBr7BH5mcbY37xcVi+0gUYAWwRkQTHtoeMMUt8\nGJMqW+4B3nMMknYBo30cj08YY34SkQXARuxsuE1UwNYN2rJBKaX8TEUu9SillMqDJn6llPIzmviV\nUsrPaOJXSik/o4lfKaX8jCZ+pQARyRSRBLdLiZ25KiLhIpJYUvtTqrgq7Dx+pYrojDEmxtdBKFUa\ndMSvVAFEJElEnhWRLSKyVkSucGwPF5HvRORnEflWRBo7toeJyCIR2ey4OE/3DxSRWY4+70tFJMRn\nP5Tye5r4lbJCcpV6bnV77LgxJhJ4BdvVE2AG8I4xJgp4D3jZsf1l4HtjTDS2343zbPEWwKvGmDZA\nKjDYyz+PUvnSM3eVAkTklDGmWh7bk4BexphdjkZ3B40xtUQkBahvjDnn2H7AGFNbRI4ADY0xGW77\nCAe+Nsa0cNx/EKhsjHna+z+ZUhfSEb9ShTP53C6KDLfbmejxNeVDmviVKtytbtdrHLd/wLUk3zBg\npeP2t8B4yF7TN7S0glTKUzrqUMoKcetcCnb9WeeUzpoi8jN21D7Use0e7IpVk7GrVzm7Wd4LzBSR\nsdiR/XjsSk5KlRla41eqAI4af6wxJsXXsShVUrTUo5RSfkZH/Eop5Wd0xK+UUn5GE79SSvkZTfxK\nKeVnNPErpZSf0cSvlFJ+5v8D86fjv9cH92gAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "plt.plot(\n",
        "    CG_frobenius_norm_of_weight,\n",
        "    color='r',\n",
        "    label='CG_frobenius_norm_of_weights')\n",
        "plt.plot(\n",
        "    SGD_frobenius_norm_of_weight,\n",
        "    color='b',\n",
        "    label='SGD_frobenius_norm_of_weights')\n",
        "plt.xlabel('Epoch')\n",
        "plt.ylabel('Frobenius norm of weights')\n",
        "plt.legend(loc=1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JGtutiXuoZyx"
      },
      "source": [
        "# 训练和验证准确率：CG 与 SGD\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "s-SNIr10o2va"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f2cd4a3f668>"
            ]
          },
          "execution_count": 14,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3xb5dnw8d9l2Zb3iO0kziK2s8h0\nIGGlpOywCg+0vITVUHjKS9nQlvVCm9KWUh76lDJaGqBQKIW2QIBSCi0tYVNwFoQMgp1gHDuJt7xl\nSff7xy3ZciInsi1FHtf38zmfc3QkHd0W5Fy613WLMQallFJqT3GxLoBSSqnBSQOEUkqpkDRAKKWU\nCkkDhFJKqZA0QCillAopPtYFiJTc3FwzefLkWBdDKaWGlNWrV9cYY/JCPTdsAsTkyZMpKSmJdTGU\nUmpIEZEventOm5iUUkqFpAFCKaVUSBoglFJKhaQBQimlVEgaIJRSSoWkAUIppVRIGiCUUkqFNGzm\nQSiloqOpo4nPaj9jS+0WtjdsZ3TqaAqzCynKLmJCxgQccY5YF1FFiQYIpRQ+46O8sZzNNZvZUrOF\nLbV221yzmcqmyl7flxCXwEFZB1GUXdQVNAqzC7u2dGf6AfwrDgxjDHVtdVQ1V9HW2cbYtLGMTRtL\ngiMh0h8EnZ3hbcnJMHNmZD8fDRBKjSiuDldXANhcs9kGgpotbK3bSrunvet1WUlZzMidwYmFJzI9\nZzrTc6czI3cGk7MmU91STWl9KWX1ZZTVl3Udf7jjQ+rb63t8Xl5KHkWjiijMKqQo4yAK0yZSmDyO\nImc++Y5M4jrc0NEB7e1770OdC+zd7r3/OJF9P97Pazrxsjuunaq4FrtJC1Vxrezc87G00Cm+vS6d\n500i35PEuM4k8t1OxnUkkt8ez7i2ePJbHYxriWNsMyS6vaFv8h5Pz8de7/7/gwYcfjh88EH4rw+T\nDJcV5RYsWGA01YaKJGMMu1t24/a6SXemk56YfmCaUzo6oK7ObrW13cdut/1V6fPZffBx0Dmv18N2\nXx1bfLvZ4qtmi6lms6lmCzXspLnrYxxGKCSb6b5RTPdlM8M3iunebKb7ssnzJSE+E/rzOjp6vYnX\ne1vYltBMqbOVsuQ2StPclKV5KM0ylGeCL6jXM6kTChqgsN5uRXX+fT0U1EOyJ8R3k5gITqfdB9/s\n97yPBT1uiTdUpfrYmeKjKsVHVaqPqlRjH6d2P65JNpgQMSW3VchvEca22H1+Sxz5zfY4ySvsTBeq\nMoTKNKhKM1SleKlM8bLL6cEbopc3tzPRBhJvCvm+VPJNKuNMGvmSwbi4DPIdWeTHZ+FMSIaEhPC2\nnBxYtCjM/8F6EpHVxpgFoZ7TGoQa0Tw+D+WN5Xxe9zmldaWU1vu3OvuruKWzpcfrUxNSyXBmkO5M\nJ8OZ0bWlJ4Z47Egho1PIaDOkt3rIaO4kw9VBRkMbyfVNSG1d6EDQ0tJLaXtqSIItObA5F7bk2uMt\nubB1FLiD/mWPaoUZNXBKLUyvgem1diuqh0TTCHFNIOUQF2dvuiLdx6HOOZ2QlLT3Pj2dbOd4spOS\nOGTP55KS6EyMp9zZRqnDRVlcI6WmjrJRtZR5qnnLvZMmb2uPv29cyhgKMyZTmF1AUc5UCnOm2H12\nIXmpedS31VPVXEVVUxU7m3d2HVc193zc5G7a67uLj4tnbNpY8tPymZyezxGpY8lPzyc/Lb9rPzZt\nLGPSxpDoSOz7/1iA1+elurWaqqYqKpsqqWr275uqqGy2+0+bq6hq2o7X7F1bGJU8inHp48hPy++5\nT88JOs4nKT6pX+ULh9Yg1LDX4m7pagrZMwh80fgFHl/3T1Wnw2nb0kcVUZRtt5S4RFz1u2hy7cbV\nVIurpQ5XWwOuDhdNnc24vK24TDsuceNyePCEMTYwzgcZbiHD6yDDl0i6OMlwJJMRn0ZGYjrpyZlk\npGSTkZZDRuZoUjNyqeqsY3NDKVsaS9nS8Dm722q6rhcfF09RZgHTR01l+qhpzMiZZpuGcqaTmzZ6\n75v8IGOMobattiswBzddldaXssO1A0P3vUqQHo8D0hLTum78wTf64Mf56fmMSh5FnAyOQZw+46Om\ntaY7eAQHk6CgsrN5J52+zr3en52UzTGTj+H5c5/v1+fvqwahAUINecE3l1BBoKq5qsfrs+LTKEoY\nwxTJocibSVF7MkXNCRTVwbhaN3H1DdDQAPX1dt/c3MsnAw4HjBplq/ijRmFGZdORm41rVCqu7BRc\nGU6a0hNxpcbjShJcieBK8NFEBy53E64Olw00QceBrdm99+fmpuQyPcf2BwT6BqbnTKcwuzDynaSD\nSLunnS8avugKGjubd5KXkmdv/kE3/rTEtFgXNWp8xkdta23Pmog/iIxOHc0PvvqDfl1XA4Qa8nzu\nDioqNlJauYHSXZsprSvl86btlLZVUurZjYuOHq8f157AlEYHRbWGol2dFNX6KKqzzSqj2kJ8QEYG\nZGdDVtbe+1Gjujd/IOg6Tk+P2i9yn/HR7G62AaSjiTFpYxiVPCoqn6VGLu2DUIOf203LhjVsW/8m\n20pLKKvaSGnHTkqdrZSmdrAt0/RoV0/wwuQG27F5VD0UNcZR1JlOkcmmMD6P5Mwce3PPyYaiEDf9\n4H1GBsQPvn8KcRLX1aehVCwMvn8Vatjy+rzsaPySso3vsm3LB5RVfEJZwza2dVZTltTGrkDrQAIw\nCdK8Doq8mcySCZwRP4ailAkUZU5mSs5UJo6ZiiM7p/tGn5w8KNvWlRrKNECoiGpob7CdjHWlbPvy\nY8q2r6Os5jO2te9ku6OJzrjuJs04H0yKi6cwMYvTk6dRmDuDwsJDKZhxJIWjp5ObkovoTV+pmNEA\nofrE7XVT3ljeNdJkW/02ynZvoWzXJrY1V1Bveg5VHNVqx7bPb3NytnMShTlTKJw0j4KZi5h0yLEk\nZGbH6C9RSu2PBgjVxRiD2+umob2BbQ3bugNAfRllDbZWUOHagY/uWaSJXpjsn+h0RD0UtDkpzDiI\nwvGzKZh2OJmLF8Ls2ZAXck10pdQgpgFiEPP6vLR52mjrbBvYfh/PtXvae5wLNbY8vzOJggZYXNVO\nYZ1/9mtTPIW5Uxk3ZT5xc+bCybNtIJg0SfsClBomNEAMEn/59C/8YNUPqG+r77pZh5oUE66k+CSS\n45PtPiGZ5Pjkrn1WUhb5afnd5+OTSe70kVxaTvLGrWRs3sbkGg+F9XakUMrkifbmP3s2zJlj91On\nDsqRP0qpyNF/4THW2tnKda9ex8NrHmb+2Pl8dcZXe9zMkxOSu272e97oe9s7453hzRLdtg1WroTn\nn4f33rP5awoL4fQrYP58GwwOPhhSUqL/RSilBp2oBggRORn4FeAAHjHG3LXH8wcBvwPygDrgQmNM\nhf85L/CJ/6XlxpgzolnWWPhk1ycsfW4pm6o3cfOim7nj2DuiOxvWGPjkExsUVq6E9evt+Xnz4Ic/\nhLPOskFBm4iUUkQxQIiIA3gQOBGoAD4SkZeMMRuDXnYP8IQx5vcichzwM+Ai/3NtxpjiaJUvlowx\nPFTyEDf84wYynZm8duFrnFh0YnQ+zOezaYADQaG01AaAo46Ce+6xQaGwMDqfrZQa0qJZgzgM+NwY\nUwYgIs8AZwLBAWImcIP/+A3ghSiWZ1Cob6vnv//63zy/6XmWFC3h9//1e8akjYnsh7jdsGqVbTp6\n8UXYudOmBD7+eLjxRjjjDBg7NrKfqZQadqIZIMYDXwY9rgAO3+M164Gzsc1QZwHpIpJjjKkFkkSk\nBPAAdxlj9goeInIZcBnApEmTIv8XRNi75e9y/vPnU9lUyf+c+D/ccOQNkcso2dICr75qawkvvwyN\njZCaCqecYmsJp50GmZmR+Syl1IgQ607q7wEPiMjFwFvADiCQGP0gY8wOESkE/i0inxhjSoPfbIxZ\nAawAm6zvwBW7b7w+Lz9752csX7Wcg7IO4r1L3mPh+IUDv3BdHfz1rzYovPaaXbRl1CgbEM4+G044\nwaagUEqpfohmgNgBTAx6PMF/rosxphJbg0BE0oCvG2Ma/M/t8O/LRGQVMB/oESCGgsqmSi58/kLe\n2P4G5885n9+c9puBJV+rqIAXXrBB4c037bKEEybAt79tA8PRR+vwU6VURETzTvIRMFVECrCBYSlw\nfvALRCQXqDPG+IBbsCOaEJFsoNUY0+F/zSLg7iiWNSr+9tnfuPjFi2ntbOWxMx9j2bxl/csttGVL\ndyfzhx/aczNm2P6Es86CBQt05JFSKuKiFiCMMR4RuQp4DTvM9XfGmE9F5A6gxBjzEnAM8DMRMdgm\npiv9bz8Y+K2I+IA4bB/Exr0+ZJDq8HRw8+s3c+9/7mXemHk8841nmJE7I/wLGANr1thO5pUrYdMm\ne37hQrjzThsUZvThekqpocXnA48XOj3g8UCn17+3j43bS32tj8oqqNwZhyM5geMvGRfxYuiCQRH2\nWe1nLH12KWt3ruXqw67m7hPvDn/N2Hfegb/8xTYhlZfb1coWL7YB4b/+CyZO3P81lFKDR+BGH7i5\n97jRB5/3dAWEJpePyl0OKmsTqaxJoLI2gcqaRCprE9hRnWDP1ybQ4e4e4LJgjpuPPu7f2tm6YNAB\n8sT6J7jib1fgjHfywrkvcOaMM8N/8yOP2H4EpxNOOgl+9CP42tfsqmVKqdjz+bpv5u5O/3Hn3jf9\n4F/9Xm/X29s7hKo6e7PfUZNgb/71SVTWJdoAUJNAZXU8TS17j2xMTTWMH2cYNw6OnCuMHy+MG0fX\nNnFi/4LD/miAiICmjiaueOUK/vDxH1h80GKeOvspJmRMCP8Cn38O111n5ym88AKkDd91dZUaNIwB\nr8/e5N2eXvb+QOD23/RD6PQIu1xJVDY4qaxLprLOSWVtIjuq46msjqdydzyVu+Ooq9/7xu90dt/k\n5y6Ek/3H48fTIwCkpwtw4PsZNUAM0OrK1Sx9bill9WUs/+pyblt8G444R/gX8Hjgm9+0E9kef1yD\ng1IDYUzPX/hBe+P24G7x0OLy0uLy0dLko6VVaGmPs1ubo/vYHU+L20lLR7w97nDQ0u6gpSOOlrbA\nJjQ0Crt3gzE9b94OB+Tn25v7lINh8fE9b/iBIJCdPbjHl2iA6CdjDPd+cC83vX4TY9LG8MayN1h8\n0OK+X+jnP4f334enn7bDVZVSPXR0QE21oXpHJ9WVHmp2eand7aO5ydDSZGhpMbS02Lmi3Td8By1t\n8bS0J3Y/bo/D6+3b3Tgpyc433XPLz7H7zMy9b/zjxtnlTxx9+J04WGmA6IfqlmoufvFiXtn6CmdO\nP5NHz3iUnJR+9BWsXg3Ll8PSpXZTapgzBlwuqKmB6mq71dRA9W4f1Tt91OzydZ+vE6rr4mhqcWCb\nVxL9W0+pyT67pRhSU/w38WzImSCkpkFqehypaWK3EDf73raUlOFxkx8IDRB99Ma2N7jg+Quoa6vj\ngVMe4IqFV/RvbkNbG1x0EYwZAw8+GPmCKnUAeDxQWxt0o6/uedx9zlBTDTW14HaH+vcShzMB8rJ8\n5GV1kpvpYcrBPvJyDLl5kDdayB0dR16+g7xxCYwaG096loPkZBCJw46GV5GmASJMHp+H5auWc+fb\ndzItZxp/v+DvzBs7r/8XvOUWO7/hH/+w6TGUGqSqqmxld80a+Phjm/sxEAjq63t/X1aGl9xMD3mZ\nHiZnulkwwR7nZnrIy/aQlwe5eXHkjbU3/tRRiUhSIiQ5wZkMcXrTjzUNEGH4ouELzn/+fN778j0u\nKb6E+065j9TE1P5f8PXX4Ve/gmuugROjlOZbqT4yBr780gaCQEBYs8YGBAARw5QCHxPGeiie4iF3\nfid5GW7y0jrITXeTl+UhL8sGgNxsDwmpTkhKtJszcJxmA0BiwuDunVWABoj9en7T81z60qV4fV7+\nePYfOW/OeQO7YH09XHyxnQl91137fblS0WAMlJXBmtWG1R/5WLMa1qwXauvsr/a4OMPMwg6WHNLK\nIUXNHDK1lXlTWklP8dkLOOL8v/T9ASApNejYCQnxGgCGAQ0QvWjrbOOG127godUPsXDcQp7++tMU\njSoa+IWvvBJ27bLrNGimVRVNxoDHi7fVzdaNHtasMaxZG8fqT+JZuzGRxmbb+ZsQL8wuaOOsI1s5\nZJrd5hzsJSUz3t70nYngHAXOsd1BIN6hAWAE0AARwqe7P2Xpc0vZsHsD3z/q+/zkuJ+Q6IjATMWn\nn7bbT34Chx468OupkcsYO0u3oxPa3dBhN0+Lm02bYc0n8az51Mmaz5JZuzWFlna7rrgz0ce8Ke2c\nf4qLQ2Z7OGSel1lzBGdGIjhTwJmlv/5VFw0QQYwxPLLmEa599VrSnem8esGrLJmyJDIXr6iAK66A\nI4+Em26KzDXV8GYMtHVAaxu0tPUIBHS4cbcbNmxLZs1nKazZmsKazzJZX5pCuz9HT2qKj+JZHi65\nsJNDD+3kkIUOZsyJJyExBUiJ7d+mhgQNEH4N7Q1c9tfL+MvGv3Bi4Yk8cdYTjE2L0LKcPp/td+js\nhCee0PUahoD2dpsvcft2aG62A2rC3RyOvr0+TgxxnZ043O3EtQe2NrthiBMwwNZdqaz+PN0Gg83J\nfPJZAp2d9pd+RobhkEPgilOEQw+FQw6BqVPjcESi5qtGLL1TAe9/+T7nPXceO5p28PMTfs73jvpe\n5JYCBXjgAfjXv+C3v4UpUyJ3XdVvbrcdsbNtmw0C27d3H2/bZod2HjjBk8D2v5hUTo4NADecTlcw\nKCgQHRWqIm7EB4gtNVs4+rGjmZg5kXe+9Q6HT9hz2ewB2rTJNimddprN1qoOCI/HBoA9b/6B4x07\nbAtOgMMBkybB5Ml2Ge/Jk7u3zExbCezL5nV78bW58bW67T5w7PHZ1xjBJ3H4EhLtlpiI15GILyEB\nnzhCXrOgwAaEiRO1i0AdGCM+QEzPnc5vTvsN/2fW/yEzKTOyF3e74cILbQK+Rx7Rf9UR5PXam3yo\nm//27bbLJyjTMnFxNtVVQYFNmjt5sj0OBIHx4/vZ8ufzQWu77SMI3jrckB704anJ/i0VUvzHOhdA\nDXIjPkAAfPvQKP2y//GPu1eGGxuh/owRwhiorOw9AJSX98y+LGKTpBUU2GW59wwAEyfahLkDKlBb\ncCDwH7e19yxEShJkptkAEAgESYkaCNSQpAEiWt5/3y4P+q1v2RXh1D55vfDJJ/Dmm/DWW3arqen5\nmvx8e7M//HCb2zBw8y8osAHA6YxQYdyd0NxqA0Bg39res00q2Wlv/qOzu4NBslPTQ6hhRQNENDQ3\n20R8kybBvffGujSDksdjK1dvvWWDwjvvQEODfa6gAE4/HQ47DAoLbRCYNCkK8wp9PjuMtLm1Z0Do\nDKqaJCbYAJCdAWkpNhCkJNmZxEoNcxogouG737V5DN58EzL2PyplJOjogJIS+5W8+Sa8956NowDT\np8M558BXv2qX4I7K0tudnr1rBS1t3bUCERsIRmXaQJCabPcJ+k9EjVz6f3+kvfwyrFhhRy4dfXSs\nSxMzbW3wwQfdNYT337dzCwBmz4Zly2wwWLw4wt0zgb6C5raeAcHd2f2ahHh78x8/2u7TUrR5SKkQ\nNEBEUnU1XHopzJsHP/pRrEtzQDU321pBoIbw4Yd2XmBcHBQXw+WX2xrC0UfbcfwR4fHYQNBVK2i1\nncc+f0K5QKdxVnrPWkHiQHqrlRo5NEBEijFw2WW2If1f/4pgj+ng1NBg+w0CncqrV9uOZocDFiyA\n66+3tYOvfMXOIxgQY6C9Y+9aQYe7+zXx8ZCWDPl5dp+WYoOD1gqU6jcNEJHy+OPwwgtwzz22DWWY\nqanpHl305puwfr29bycm2lFFN99sawhHHmmnfQyIuxPqXdDY7K8VtIHX1/18ShJkpEJaXs9agQ4l\nVSqiNEBEwrZtcO21cMwx9qfzMFBV1R0M3noLPv3Unk9OtkFg+XJbQzj88AiMLjIGXM1Q54K6Rls7\nAFsdSUuGsbndgSAlWUcQKXWAaIAYKK/X9riK2FrEEGzS8HptAHj/fdux/O67sHWrfS4tzTYTXXCB\nrSEsWGBrDQPW7ob6RhsU6l3d054z02DyeBjlH1aqtQKlYkYDxED94hfw9ts2S+tBB8W6NGGprraB\nILB9+GH3kNPcXFtD+L//19YQ5s+PUPJZn882GdU12q3VP6TJmQB52XZ4aXa6ZrpVahDRf40DsX49\n3HYbfOMbNufSINTZaWcoB2oH778PpaX2OYfDjjBatswGhSOOsBPTIvajva29u9moockGCRFbSxib\na4NCSpLWEpQapDRA9Fd7uw0Kubnw0EOD5ia3c2d3MPjgA/joIzsnAex8g0Dt4IgjbGbQlEiuG+P1\n2kBQ5286au+w55Od/oCQYYecOhwR/FClVLRogOiv226DDRvg73+P4MD+vnG7Yd26nrWDL76wzyUk\n2HUCLrusu3YwaVKE45gxtqko0GzU2GzPxcXZQDBhjA0KyUkR/FCl1IGiAaI/3ngD/vd/7RKiJ598\nwD62oqJnMFizxqawAJue4ogj4JprbECYPx+SonFf9nhsp3Kg6SgwQzk12c5MHpVpm5CGYGe9Uqon\nDRB91dhoG+2nToW7747ax7S32wAQHBB27LDPOZ12NNFVV3XXDsaPj1JBjLHDTgPNRi5/b7bDYRPY\njcqwQcGpS1sqNdxogOirq6+2CxW8955d/CVC3G47z+7dd20wWLfOdjCDzW66eHF3MJg3L0JDTXst\njH+iWl2j3Qeym6anwKR8GxQy0gZNv4tSKjo0QPTFs8/Ck0/CD39oc1FHyL//DVdeCZs3207jhQvh\nhhtsQDj88AO01pDPB7UNUFltO5rBJrXL9tcQsjM0h5FSI4wGiHBVVdnhPwsXwv/7fxG55I4d8L3v\nwTPP2FrCiy/Cqace4KkAbe1QVQM7a2xNwZkIB+VDTpZOVFNqhNMAEQ5j4JJL7HjRJ58c4NqVtuno\nvvtsuorOTru/8cYoLIjTm1C1hZwsyPfPTdCgoJQiygFCRE4GfgU4gEeMMXft8fxBwO+APKAOuNAY\nU+F/bhlwm/+lPzHG/D6aZd2nhx6CV1+FBx+0q9sMwKpVtjlp40ZbW7jvPigqikwx9ytUbWHyODtH\nQTuZlVJ7iFqAEBEH8CBwIlABfCQiLxljNga97B7gCWPM70XkOOBnwEUiMgr4IbAAMMBq/3vro1Xe\nXn32mV0hbskS+M53+n2Zqir4/vfhqafsEpovvghf+9oB+LGutQWlVD9FswZxGPC5MaYMQESeAc4E\nggPETOAG//EbwAv+4yXAP40xdf73/hM4GXg6iuXdW2ennS2dnAy/+12/bqYeDzzwAPzgB3bOwu23\n29TYEZ3BHIrWFpRSAxTNADEe+DLocQVw+B6vWQ+cjW2GOgtIF5GcXt4brZH+vbvzTpur4i9/gXHj\n+vz2t9+2zUmffGLn091/P0yZEoVyBmhtQSkVQbHupP4e8ICIXAy8BewAvOG+WUQuAy4DmDRpUmRL\n9uGH8OMfw0UX2WR8fbBzp+10fvJJm95i5Uo488wo3p+1tqCUioJoBogdwMSgxxP857oYYyqxNQhE\nJA34ujGmQUR2AMfs8d5Ve36AMWYFsAJgwYIFJmIlb2mxTUvjxtmf/WHyeODXv7bNSG1tcOutdovg\nfLpugdpCVY2dzAZaW1BKRVQ0A8RHwFQRKcAGhqXA+cEvEJFcoM4Y4wNuwY5oAngNuFNEsv2PT/I/\nf2DceKNdMeff/w57QeV337XNSevXw4kn2n6HadOiULaQ8xbG2cCgtQWlVARFLUAYYzwichX2Zu8A\nfmeM+VRE7gBKjDEvYWsJPxMRg21iutL/3joR+TE2yADcEeiwjrpXX7XVgBtugGOP3e/Ld++Gm26y\ni8lNmGAnW599doR/wGttQSkVA2JM5FpmYmnBggWmpKRkYBeprYU5c2DUKCgp2Wc6VK/XTo+47Tbb\nIvXd79oJ1mlpAytCD6FqC2NztbaglIoYEVltjFkQ6rlYd1IPHsbA5ZdDTQ288so+g8MHH9hM32vX\nwvHH2+akGTMiVA6tLSilBgkNEAFPPWXbh+66y67DGUJ1NdxyCzz6qO2//tOf4JxzInTPNga2V0JV\ntfYtKKUGBQ0QAOXltof5K1+x2fP24PXCww/bEUlNTXZG9O23Q3p6BMtQXQ/lVbaWMC5PawtKqZjT\nAOHz2QWAfD544om91kv+8EMbO0pK4JhjbDqmmTOjUI76Roh3wOwpGhiUUoOCrgtZWmrXlr7vPptz\n26+21q7nfMQRNi33H/9oR71GJTgYA/VNkJWhwUEpNWhoDWLqVNiyBbLtlAufz/Yx3HyzXV30+uvt\n+kAZGVEsQ1sHdLjtam1KKTVIaIAAO6wV24x05ZW2WWnxYtucNHv2Afj8wGil7Eh2aiil1MBoExNQ\nV2czeR92GHzxhc2htGrVAQoOYANEUiIkOQ/QByql1P6N+BrE1q1w1FE2SFxzDfzoR2Fn14gMY2zm\n1dHZ2v+glBpURnyAKCqCpUvhv/8b5s2LQQGaWuw42qxodnIopVTf7TdAiMjVwB9isprbARAX16eE\nrZGn/Q9KqUEqnD6IMdjlQv8sIieLaDtIRNW7IC0FEhJiXRKllOphvwHCGHMbMBV4FLgY2Coid4pI\nUZTLNvx5veBqgWxtXlJKDT5hjWIyNuXrTv/mAbKBZ0Xk7iiWbfhraLad1BoglFKDUDh9ENcC3wRq\ngEeA7xtjOkUkDtgK3BjdIg5j9S47cikjkjnClVIqMsIZxTQKONsY80XwSWOMT0ROj06xRogGF2Sm\ngUOnoyilBp9w7kx/B7pWcxORDBE5HMAYsylaBRv23J3Q0qbNS0qpQSucAPEboDnocbP/nBqIruGt\nGiCUUoNTOAFCTNC6pMYYHzrBbuDqXTa9d1pKrEuilFIhhRMgykTkGhFJ8G/XAmXRLtiwZoztf8jW\n9N5KqcErnABxOXAUsAOoAA4HLotmoYa9tnbo6NT0GkqpQW2/TUXGmN3A0gNQlpGjvsnutf9BKTWI\nhTMPIgm4FJgFJAXOG2MuiWK5hrd6l03tnazpvZVSg1c4TUxPAmOBJcCbwASgKZqFGta6+h80OZ9S\nanALJ0BMMcbcDrQYY34PnFI5r6YAABnLSURBVIbth1D94WoBr0+bl5RSg144AaLTv28QkdlAJjA6\nekUa5gLzH7SDWik1yIUzn2GFiGQDtwEvAWnA7VEt1XDWEEjvrVNJlFKD2z7vUv6EfC7/YkFvAYUH\npFTDVSC994QxsS6JUkrt1z6bmPyzpjVba6Q0NGl6b6XUkBFOH8TrIvI9EZkoIqMCW9RLNhzVuyBO\nbAZXpZQa5MJpCD/Xv78y6JxBm5v6rt4Fmel2IWyllBrkwplJXXAgCjLsdbihtR3G5MS6JEopFZZw\nZlJ/M9R5Y8wTkS/OMNYQSK+RGdtyKKVUmMJpYloYdJwEHA+sATRA9EW9C+LjIS051iVRSqmwhNPE\ndHXwYxHJAp6JWomGI2NsgMhO1/TeSqkhoz+9pS2A9kv0RWu7XWJUh7cqpYaQcPog/oodtQQ2oMwE\n/hzNQg07uryoUmoICqcP4p6gYw/whTGmIpyLi8jJwK8AB/CIMeauPZ6fBPweyPK/5mZjzCsiMhnY\nBGzxv/QDY8zl4XzmoNTgT++dpOm9lVJDRzgBohyoMsa0A4hIsohMNsZs39ebRMQBPAiciF2J7iMR\neckYszHoZbcBfzbG/EZEZgKvAJP9z5UaY4r79NcMRj6fHcE0Woe3KqWGlnD6IP4C+IIee/3n9ucw\n4HNjTJkxxo3t2D5zj9cYINDukglUhnHdoaVJ03srpYamcAJEvP8GD4D/ODGM940Hvgx6XOE/F2w5\ncKGIVGBrD8EjpgpEZK2IvCkiR4f6ABG5TERKRKSkuro6jCLFQGB50SxdIEgpNbSEEyCqReSMwAMR\nOROoidDnnwc8boyZAJwKPOnPIFsFTDLGzAduAP4oInv9BDfGrDDGLDDGLMjLy4tQkSKs3gXpmt5b\nKTX0hHPXuhx4SkQe8D+uAELOrt7DDmBi0OMJ/nPBLgVOBjDGvO9f/zrXGLMb6PCfXy0ipcA0oCSM\nzx08PF5wNcPEsbEuiVJK9Vk4E+VKgSNEJM3/uDnMa38ETBWRAmxgWAqcv8dryrEzsx8XkYOxM7Wr\nRSQPqDPGeEWkEJgKlIX5uYNHYyC9hvY/KKWGnv02MYnInSKSZYxpNsY0i0i2iPxkf+8zxniAq4DX\nsENW/2yM+VRE7ghqsvou8G0RWQ88DVxsjDHAYuBjEVkHPAtcboyp69+fGEP1Lpu5VdN7K6WGILH3\n4328QGStvy8g+NwaY8whUS1ZHy1YsMCUlAyyFqiPNoAzEeZOi3VJlFIqJBFZbYxZEOq5cDqpHSLS\nNcNLRJIBnfG1P4H03tq8pJQaosLppH4K+JeIPAYIcDF29rPaF02voZQa4sLppP65v4/gBOzEtteA\ng6JdsCGvockObU3V9N5KqaEp3Gyuu7DB4RzgOGyns+pNIL13Voam91ZKDVm91iBEZBp2Itt52Ilx\nf8J2ah97gMo2dHWl99bZ00qpoWtfTUybgbeB040xnwOIyPUHpFRDnfY/KKWGgX01MZ2NTXnxhog8\nLCLHYzup1f7UuyBZ03srpYa2XgOEMeYFY8xSYAbwBnAdMFpEfiMiJx2oAg45gfTeWntQSg1x++2k\nNsa0GGP+aIz5Gjaf0lrgpqiXbKhqarFBQgOEUmqI69Oa1MaYen8G1eOjVaAhL9D/kKkd1Eqpoa1P\nAUKFod4F6ama3lspNeRpgIgkjwdcLdq8pJQaFjRARFKDPxO6zn9QSg0DGiAiqcGf3jtD03srpYY+\nDRCRVO+yaz/E6deqlBr69E4WKZreWyk1zGiAiBRNr6GUGmY0QERKvUvTeyulhhUNEJEQSO+drem9\nlVLDhwaISGhth06PNi8ppYYVDRCRUN9o91kaIJRSw4fmg4iE+iZIToKkxFiXRKlhobOzk4qKCtrb\n22NdlGEjKSmJCRMmkJCQEPZ7NEAMVCC999icWJdEqWGjoqKC9PR0Jk+ejGi/3oAZY6itraWiooKC\ngoKw36dNTAPl0vTeSkVae3s7OTk5GhwiRETIycnpc41MA8RABeY/ZGn+JaUiSYNDZPXn+9QAMVCB\n9N7x2lqnlBpeNEAMhMdjV5DT5iWl1DCkAWIgGprsXgOEUsPSzp07Wbp0KUVFRRx66KGceuqpfPbZ\nZ2zdupXTTz+96/yxxx7LW2+91et1Vq1axXvvvdfnzy8pKeGaa64ZyJ8wINouMhD1gfTeqbEuiVIq\nwowxnHXWWSxbtoxnnnkGgPXr17Nr1y4uvfRS7rnnHs444wwANmzYQElJCYsXLw55rVWrVpGWlsZR\nRx2113Mej4f4XpqoFyxYwIIFCyL0F/WdBoiBqG+CLE3vrVRUXXcdrFsX2WsWF8O99+7zJW+88QYJ\nCQlcfvnlXefmzZvHo48+ypFHHtkVHABmz57N7NmzQ15n+/btPPTQQzgcDv7whz9w//338+ijj5KU\nlMTatWtZtGgRS5cu5dprr6W9vZ3k5GQee+wxpk+fzqpVq7jnnnt4+eWXWb58OeXl5ZSVlVFeXs51\n110X9dqFBoj+andDWzuMy411SZRSUbBhwwYOPfTQvc5/+umnHHLIIWFfZ/LkyVx++eWkpaXxve99\nD4BHH32UiooK3nvvPRwOBy6Xi7fffpv4+Hhef/11br31Vp577rm9rrV582beeOMNmpqamD59Ot/5\nznf6NPGtrzRA9FdDYHir9j8oFVX7+aUfa2eddRZbt25l2rRpPP/882G/75xzzsHhcADQ2NjIsmXL\n2Lp1KyJCZ2dnyPecdtppOJ1OnE4no0ePZteuXUyYMCEif0co2jbSX5reW6lhbdasWaxevTrk+TVr\n1nQ9XrlyJY8//jh1dXV9un5qanff5e23386xxx7Lhg0b+Otf/9rrhDan09l17HA48Hg8ffrMvtIA\n0R+a3lupYe+4446jo6ODFStWdJ37+OOPmTZtGu+++y4vvfRS1/nW1tZ9Xis9PZ2mpqZen29sbGT8\n+PEAPP744wMreARpgOiPljZN763UMCcirFy5ktdff52ioiJmzZrFLbfcwtixY3n55Zd56KGHKCws\n5Mgjj+QnP/kJt912W6/X+trXvsbKlSspLi7m7bff3uv5G2+8kVtuuYX58+dHvVbQF2KMiXUZImLB\nggWmpKTkwHzYlzuhrAKOmAtOzeCqVKRt2rSJgw8+ONbFGHZCfa8istoYE3IsbVRrECJysohsEZHP\nReTmEM9PEpE3RGStiHwsIqcGPXeL/31bRGRJNMvZZ/UuSEnS4KCUGtaiNopJRBzAg8CJQAXwkYi8\nZIzZGPSy24A/G2N+IyIzgVeAyf7jpcAsYBzwuohMM8Z4o1XesPl80NgMY3V4q1Kqp8cee4xf/epX\nPc4tWrSIBx98MEYlGphoDnM9DPjcGFMGICLPAGcCwQHCAIGG/Eyg0n98JvCMMaYD2CYin/uv934U\nyxseV7Om91ZKhfStb32Lb33rW7EuRsREs4lpPPBl0OMK/7lgy4ELRaQCW3u4ug/vjY16/0iErLTY\nlkMppaIs1qOYzgMeN8ZMAE4FnhSRsMskIpeJSImIlFRXV0etkD3Uu2zuJU3vrZQa5qIZIHYAE4Me\nT/CfC3Yp8GcAY8z7QBKQG+Z7McasMMYsMMYsyMvLi2DRe9Gp6b2VUiNHNAPER8BUESkQkURsp/NL\ne7ymHDgeQEQOxgaIav/rloqIU0QKgKnAh1Esa3gaA81LGiCUUsNf1AKEMcYDXAW8BmzCjlb6VETu\nEJFAGsTvAt8WkfXA08DFxvoUW7PYCLwKXDkoRjDVu8Ch6b2VGilivR4E2Gywf/zjH/v7JwxIVBvS\njTGvYDufg8/9IOh4I7Col/f+FPhpNMvXZ/UuyEzX9N5KjQAHaj2I/QkEiPPPP7//f0w/aU9ruNo7\noK0Dxo2OdUmUGlGue/U61u2M7HoQxWOLuffk2K0HMWPGDC6//HLKy8sBuPfee1m0aBFvvvkm1157\nLWBTfbz11lvcfPPNbNq0ieLiYpYtW8b1118/0D8/bBogwlWvy4sqNZJEcz2I888/n+uvv56vfOUr\nlJeXs2TJEjZt2sQ999zDgw8+yKJFi2hubiYpKYm77rqra9GgA00DRLjqXZCYYFNsKKUOmP390o+1\n/qwH8frrr7NxY/ecYZfLRXNzM4sWLeKGG27gggsu4Oyzz47qWg/h0Mb0cBhjFwjKStf03kqNENFc\nD8Ln8/HBBx+wbt061q1bx44dO0hLS+Pmm2/mkUceoa2tjUWLFrF58+aI/C39pQEiHJreW6kRJ5rr\nQZx00kncf//9XY/X+dfcLi0tZc6cOdx0000sXLiQzZs373ctiWjSABGOev/yohoglBoxorkexH33\n3UdJSQlz585l5syZPPTQQ4DtrJ49ezZz584lISGBU045hblz5+JwOJg3bx6//OUvD9SfD+h6EOH5\n+DPocMPC0KMUlFKRpetBRMegWg9iWAik99bag1JqhNFRTPvT6E/vrek1lFL7oetBjDQN/v6HrPTY\nlkMpNejpehAjTb0LMtIg3hHrkiil1AGlAWJfOj3Q1ArZWntQSo08GiD2pUHTayilRi4NEPsSSO+d\nrum9lVIjjwaIfQmk19D03kqNSD/96U+ZNWsWc+fOpbi4mP/85z94PB5uvfVWpk6dSnFxMcXFxfz0\np90rEzgcDoqLi5k1axbz5s3jF7/4BT6fr9fPWLduHa+88kqvz/emsrKSb3zjG/36u8Klo5h6o+m9\nlRocPi+H5n2nsuiztBSYMmmfL3n//fd5+eWXWbNmDU6nk5qaGtxuN7fddhs7d+7kk08+ISkpiaam\nJn7xi190vS85Obkrdcbu3bs5//zzcblc/OhHPwr5OevWraOkpIRTTz11r+c8Hg/x8aFv0+PGjePZ\nZ58N9y/uFw0QvdH0GkqNaFVVVeTm5uJ0OgHIzc2ltbWVhx9+mO3bt5OUZDM7p6ens3z58pDXGD16\nNCtWrGDhwoUsX74c2SPZp9vt5gc/+AFtbW2888473HLLLWzatInS0lLKysqYNGkSP/vZz7joooto\naWkB4IEHHuCoo45i+/btnH766WzYsIHHH3+cl156idbWVkpLSznrrLO4++67B/wdaIDojab3Vmpw\n2M8v/Wg56aSTuOOOO5g2bRonnHAC5557LtnZ2UyaNIn09PBHNhYWFuL1etm9ezdjxozp8VxiYiJ3\n3HEHJSUlPPDAAwAsX76cjRs38s4775CcnExrayv//Oc/SUpKYuvWrZx33nmESiu0bt061q5di9Pp\nZPr06Vx99dVMnDhxQN+BNq6HYowdwZSdoem9lRqh0tLSWL16NStWrCAvL49zzz2XVatW9XjNY489\nRnFxMRMnTuTLL7+M2GefccYZJCcnA9DZ2cm3v/1t5syZwznnnNNjHYlgxx9/PJmZmSQlJTFz5ky+\n+OKLAZdDaxChNGt6b6WU7XA+5phjOOaYY5gzZw6//e1vKS8vp6mpifT09K6Z07Nnz8br9Ya8RllZ\nGQ6Hg9Gjw+/PTE3tHjn5y1/+kjFjxrB+/Xp8Pl9X09aeAk1hgXJ7PJ6wP683WoMIpb7R7jW9hlIj\n1pYtW9i6dWvX43Xr1jF9+nQuvfRSrrrqKtrb2wHwer243e6Q16iurubyyy/nqquu2qv/IWB/6z00\nNjaSn59PXFwcTz75ZK+BKBq0BhFKQ5Pte3AmxrokSqkYaW5u5uqrr6ahoYH4+HimTJnCihUryMzM\n5Pbbb2f27Nmkp6eTnJzMsmXLGDduHABtbW0UFxfT2dlJfHw8F110ETfccEOvn3Psscdy1113UVxc\nzC233LLX81dccQVf//rXeeKJJzj55JN71C6iTdeD2JPPB++uhfy8mHWOKTXS6XoQ0aHrQQxUYzP4\njPY/KKVGPG1i2lO9y45cytT+B6VU5Lz22mvcdNNNPc4VFBSwcuXKGJVo/zRA7KnBBRmpmt5bKRVR\nS5YsYcmSJbEuRp9oE1OwQHpvXT1OKaU0QPTQoOk1lFIqQANEsPomm947Q9N7K6WUBohg9S7bvKTp\nNZRSSgNEl7YOm+JblxdVSvkN5vUgABoaGvj1r3/dr/eGQ0cxBWh6b6UGpeuuA//yChFTXAz33rvv\n1wyG9SD2JxAgrrjiij6/Nxxagwho8Kf3Ttb03kqp0OtBZGVl8fDDD3P//ff3aT2IBx54gFBZKwLr\nQfzpT3+iuLiYP/3pT7S0tHDJJZdw2GGHMX/+fF588UUAPv30Uw477DCKi4uZO3cuW7du5eabb6a0\ntJTi4mK+//3vR/w70BoE2PTe9S7IydL+B6UGmf390o+WWK0Hceutt3Lcccfxu9/9joaGBg477DBO\nOOEEHnroIa699louuOAC3G43Xq+Xu+66iw0bNnTVWCJNaxBglzP0eLV5SSnVJVbrQfzjH//oSt53\nzDHH0N7eTnl5OUceeSR33nknP//5z/niiy+61ouIJq1BgPY/KKVCisV6EMYYnnvuOaZPn97j/MEH\nH8zhhx/O3/72N0499VR++9vfUlhYOOC/cV+0BgE2QKQm2z4IpZQidutBLFmyhPvvv7+rz2Lt2rWA\nDTSFhYVcc801nHnmmXz88cf7XUtioKIaIETkZBHZIiKfi8jNIZ7/pYis82+fiUhD0HPeoOdeiloh\nvT6bwVXTayilgjQ3N7Ns2TJmzpzJ3Llz2bhxI8uXL+enP/0p+fn5zJ49m/nz53P00UeHXA9i1qxZ\nnHDCCZx00kn88Ic/7PVzjj32WDZu3NjVSX377bfT2dnJ3LlzmTVrFrfffjsAf/7zn5k9ezbFxcVs\n2LCBb37zm+Tk5LBo0SJmz54dlU7qqK0HISIO4DPgRKAC+Ag4zxgTckFVEbkamG+MucT/uNkYkxbu\n5/V7PYgON5RVwNhcbWJSapDQ9SCiYzCtB3EY8LkxpswY4waeAc7cx+vPA56OYnlCcybCwYUaHJRS\nag/R7KQeDwR361cAh4d6oYgcBBQA/w46nSQiJYAHuMsY80KI910GXAYwaZKu/qaUGrx0PYj+Wwo8\na4wJHgZwkDFmh4gUAv8WkU+MMaXBbzLGrABWgG1iOnDFVUpFmzGm147doSjW60H0pzshmk1MO4CJ\nQY8n+M+FspQ9mpeMMTv8+zJgFTA/8kVUSg1GSUlJ1NbW9uumpvZmjKG2trZr9ne4olmD+AiYKiIF\n2MCwFDh/zxeJyAwgG3g/6Fw20GqM6RCRXGARcHcUy6qUGkQmTJhARUUF1dXVsS7KsJGUlMSECRP6\n9J6oBQhjjEdErgJeAxzA74wxn4rIHUCJMSYwdHUp8Izp+VPhYOC3IuLD1nLu6m30k1Jq+ElISKCg\noCDWxRjxojbM9UDr9zBXpZQawWI1zFUppdQQpgFCKaVUSMOmiUlEqoEvBnCJXKAmQsUZ6vS76Em/\nj570++g2HL6Lg4wxeaGeGDYBYqBEpKS3driRRr+LnvT76Em/j27D/bvQJiallFIhaYBQSikVkgaI\nbitiXYBBRL+LnvT76Em/j27D+rvQPgillFIhaQ1CKaVUSBoglFJKhTTiA8T+lkUdSURkooi8ISIb\nReRTEbk21mWKNRFxiMhaEXk51mWJNRHJEpFnRWSziGwSkSNjXaZYEpHr/f9ONojI0yLSt1SpQ8CI\nDhD+ZVEfBE4BZgLnicjM2JYqpjzAd40xM4EjgCtH+PcBcC2wKdaFGCR+BbxqjJkBzGMEfy8iMh64\nBlhgjJmNTUi6NLalirwRHSDo+7Kow5oxpsoYs8Z/3IS9AYyPbaliR0QmAKcBj8S6LLEmIpnAYuBR\nAGOM2xjTENtSxVw8kCwi8UAKUBnj8kTcSA8QoZZFHbE3xGAiMhm7SNN/YluSmLoXuBHwxbogg0AB\nUA085m9ye0REUmNdqFjxL2h2D1AOVAGNxph/xLZUkTfSA4QKQUTSgOeA64wxrliXJxZE5HRgtzFm\ndazLMkjEA4cAvzHGzAdagBHbZ+df1OxMbOAcB6SKyIWxLVXkjfQA0ZdlUUcEEUnABoenjDHPx7o8\nMbQIOENEtmObHo8TkT/EtkgxVQFUGGMCNcpnsQFjpDoB2GaMqTbGdALPA0fFuEwRN9IDRNeyqCKS\niO1kemk/7xm2xK4Q/yiwyRjzv7EuTywZY24xxkwwxkzG/n/xb2PMsPuFGC5jzE7gSxGZ7j91PDCS\nV3ksB44QkRT/v5vjGYad9tFck3rQ621Z1BgXK5YWARcBn4jIOv+5W40xr8SwTGrwuBp4yv9jqgz4\nVozLEzPGmP+IyLPAGuzov7UMw7QbmmpDKaVUSCO9iUkppVQvNEAopZQKSQOEUkqpkDRAKKWUCkkD\nhFJKqZA0QCjVByLiFZF1QVvEZhOLyGQR2RCp6yk1UCN6HoRS/dBmjCmOdSGUOhC0BqFUBIjIdhG5\nW0Q+EZEPRWSK//xkEfm3iHwsIv8SkUn+82NEZKWIrPdvgTQNDhF52L/OwD9EJDlmf5Qa8TRAKNU3\nyXs0MZ0b9FyjMWYO8AA2EyzA/cDvjTFzgaeA+/zn7wPeNMbMw+Y0Cszgnwo8aIyZBTQAX4/y36NU\nr3QmtVJ9ICLNxpi0EOe3A8cZY8r8CQ93GmNyRKQGyDfGdPrPVxljckWkGphgjOkIusZk4J/GmKn+\nxzcBCcaYn0T/L1Nqb1qDUCpyTC/HfdERdOxF+wlVDGmAUCpyzg3av+8/fo/upSgvAN72H/8L+A50\nrXudeaAKqVS49NeJUn2THJTpFuwazYGhrtki8jG2FnCe/9zV2FXYvo9dkS2QAfVaYIWIXIqtKXwH\nuzKZUoOG9kEoFQH+PogFxpiaWJdFqUjRJiallFIhaQ1CKaVUSFqDUEopFZIGCKWUUiFpgFBKKRWS\nBgillFIhaYBQSikV0v8HucVv/ez1EcAAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "plt.plot(history_cg.history['accuracy'], color='r', label='CG_train')\n",
        "plt.plot(history_cg.history['val_accuracy'], color='g', label='CG_test')\n",
        "plt.plot(history_sgd.history['accuracy'], color='pink', label='SGD_train')\n",
        "plt.plot(history_sgd.history['val_accuracy'], color='b', label='SGD_test')\n",
        "plt.xlabel('Epoch')\n",
        "plt.ylabel('Accuracy')\n",
        "plt.legend(loc=4)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "optimizers_conditionalgradient.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
